Goto

Collaborating Authors

 human designer


Automating eHMI Action Design with LLMs for Automated Vehicle Communication

Xia, Ding, Gui, Xinyue, Gao, Fan, Li, Dongyuan, Colley, Mark, Igarashi, Takeo

arXiv.org Artificial Intelligence

The absence of explicit communication channels between automated vehicles (AVs) and other road users requires the use of external Human-Machine Interfaces (eHMIs) to convey messages effectively in uncertain scenarios. Currently, most eHMI studies employ predefined text messages and manually designed actions to perform these messages, which limits the real-world deployment of eHMIs, where adaptability in dynamic scenarios is essential. Given the generalizability and versatility of large language models (LLMs), they could potentially serve as automated action designers for the message-action design task. To validate this idea, we make three contributions: (1) We propose a pipeline that integrates LLMs and 3D renderers, using LLMs as action designers to generate executable actions for controlling eHMIs and rendering action clips. (2) We collect a user-rated Action-Design Scoring dataset comprising a total of 320 action sequences for eight intended messages and four representative eHMI modalities. The dataset validates that LLMs can translate intended messages into actions close to a human level, particularly for reasoning-enabled LLMs. (3) We introduce two automated raters, Action Reference Score (ARS) and Vision-Language Models (VLMs), to benchmark 18 LLMs, finding that the VLM aligns with human preferences yet varies across eHMI modalities.


AGENT: An Aerial Vehicle Generation and Design Tool Using Large Language Models

Samplawski, Colin, Cobb, Adam D., Jha, Susmit

arXiv.org Artificial Intelligence

Computer-aided design (CAD) is a promising application area for emerging artificial intelligence methods. Traditional workflows for cyberphysical systems create detailed digital models which can be evaluated by physics simulators in order to narrow the search space before creating physical prototypes. A major bottleneck of this approach is that the simulators are often computationally expensive and slow. Recent advancements in AI methods offer the possibility to accelerate these pipelines. We use the recently released AircraftVerse dataset, which is especially suited for developing and evaluating large language models for designs. AircraftVerse contains a diverse set of UAV designs represented via textual design trees together with detailed physics simulation results. Following the recent success of large language models (LLMs), we propose AGENT (Aircraft GENeraTor). AGENT is a comprehensive design tool built on the CodeT5+ LLM which learns powerful representations of aircraft textual designs directly from JSON files. We develop a curriculum of training tasks which imbues a single model with a suite of useful features. AGENT is able to generate designs conditioned on properties of flight dynamics (hover time, maximum speed, etc.). Additionally, AGENT can issue evaluations of designs allowing it to act as a surrogate model of the physics simulation that underlies the AircraftVerse dataset. We present a series of experiments which demonstrate our system's abilities. We are able to achieve strong performance using the smallest member of the CodeT5+ family (220M parameters). This allows for a flexible and powerful system which can be executed on a single GPU enabling a clear path toward future deployment.


Explainable Fuzzy Neural Network with Multi-Fidelity Reinforcement Learning for Micro-Architecture Design Space Exploration

Fan, Hanwei, Wang, Ya, Li, Sicheng, Liang, Tingyuan, Zhang, Wei

arXiv.org Artificial Intelligence

With the continuous advancement of processors, modern micro-architecture designs have become increasingly complex. The vast design space presents significant challenges for human designers, making design space exploration (DSE) algorithms a significant tool for $\mu$-arch design. In recent years, efforts have been made in the development of DSE algorithms, and promising results have been achieved. However, the existing DSE algorithms, e.g., Bayesian Optimization and ensemble learning, suffer from poor interpretability, hindering designers' understanding of the decision-making process. To address this limitation, we propose utilizing Fuzzy Neural Networks to induce and summarize knowledge and insights from the DSE process, enhancing interpretability and controllability. Furthermore, to improve efficiency, we introduce a multi-fidelity reinforcement learning approach, which primarily conducts exploration using cheap but less precise data, thereby substantially diminishing the reliance on costly data. Experimental results show that our method achieves excellent results with a very limited sample budget and successfully surpasses the current state-of-the-art. Our DSE framework is open-sourced and available at https://github.com/fanhanwei/FNN\_MFRL\_ArchDSE/\ .


Google says its AI designs chips better than humans – experts disagree

New Scientist

Can AI design a chip that's more efficient than human-made ones? Google DeepMind says its artificial intelligence has helped design chips that are already being used in data centres and even smartphones. But some chip design experts are sceptical of the company's claims that such AI can plan new chip layouts better than humans can. The newly named AlphaChip method can design "superhuman chip layouts" in hours, rather than relying on weeks or months of human effort, said Anna Goldie and Azalia Mirhoseini, researchers at Google DeepMind, in a blog post. This AI approach uses reinforcement learning to figure out the relationships among chip components and gets rewarded based on the final layout quality.


Reading Users' Minds from What They Say: An Investigation into LLM-based Empathic Mental Inference

Zhu, Qihao, Chong, Leah, Yang, Maria, Luo, Jianxi

arXiv.org Artificial Intelligence

In human-centered design, developing a comprehensive and in-depth understanding of user experiences, i.e., empathic understanding, is paramount for designing products that truly meet human needs. Nevertheless, accurately comprehending the real underlying mental states of a large human population remains a significant challenge today. This difficulty mainly arises from the trade-off between depth and scale of user experience research: gaining in-depth insights from a small group of users does not easily scale to a larger population, and vice versa. This paper investigates the use of Large Language Models (LLMs) for performing mental inference tasks, specifically inferring users' underlying goals and fundamental psychological needs (FPNs). Baseline and benchmark datasets were collected from human users and designers to develop an empathic accuracy metric for measuring the mental inference performance of LLMs. The empathic accuracy of inferring goals and FPNs of different LLMs with varied zero-shot prompt engineering techniques are experimented against that of human designers. Experimental results suggest that LLMs can infer and understand the underlying goals and FPNs of users with performance comparable to that of human designers, suggesting a promising avenue for enhancing the scalability of empathic design approaches through the integration of advanced artificial intelligence technologies. This work has the potential to significantly augment the toolkit available to designers during human-centered design, enabling the development of both large-scale and in-depth understanding of users' experiences.


A Controllable Co-Creative Agent for Game System Design

Agarwal, Rohan, Lin, Zhiyu, Riedl, Mark

arXiv.org Artificial Intelligence

Many advancements have been made in procedural content generation for games, and with mixed-initiative co-creativity, have the potential for great benefits to human designers. However, co-creative systems for game generation are typically limited to specific genres, rules, or games, limiting the creativity of the designer. We seek to model games abstractly enough to apply to any genre, focusing on designing game systems and mechanics, and create a controllable, co-creative agent that can collaborate on these designs. We present a model of games using state-machine-like components and resource flows, a set of controllable metrics, a design evaluator simulating playthroughs with these metrics, and an evolutionary design balancer and generator. We find this system to be both able to express a wide range of games and able to be human-controllable for future co-creative applications.


AI-Powered Graphic Design: Exploring the Intersection of Creativity and Technology

#artificialintelligence

Welcome to the exciting world of graphic design where creativity has no more limits. AI has been rapidly evolving in recent years, and it has already started to revolutionise the way we approach design. This can save designers a lot of time and effort, allowing them to focus on the more creative aspects of their work. One of the most significant advantages of using AI in graphic design is that it can help designers generate new and innovative ideas. AI-powered tools can analyze vast amounts of data, identify patterns, and offer suggestions that human designers might not have thought of otherwise.


Creative Wand: A System to Study Effects of Communications in Co-Creative Settings

Lin, Zhiyu, Agarwal, Rohan, Riedl, Mark

arXiv.org Artificial Intelligence

Recent neural generation systems have demonstrated the potential for procedurally generating game content, images, stories, and more. However, most neural generation algorithms are "uncontrolled" in the sense that the user has little say in creative decisions beyond the initial prompt specification. Co-creative, mixed-initiative systems require user-centric means of influencing the algorithm, especially when users are unlikely to have machine learning expertise. The key to co-creative systems is the ability to communicate ideas and intent from the user to the agent, as well as from the agent to the user. Key questions in co-creative AI include: How can users express their creative intentions? How can creative AI systems communicate their beliefs, explain their moves, or instruct users to act on their behalf? When should creative AI systems take initiative? The answer to such questions and more will enable us to develop better co-creative systems that make humans more capable of expressing their creative intents. We introduce CREATIVE-WAND, a customizable framework for investigating co-creative mixed-initiative generation. CREATIVE-WAND enables plug-and-play injection of generative models and human-agent communication channels into a chat-based interface. It provides a number of dimensions along which an AI generator and humans can communicate during the co-creative process. We illustrate the CREATIVE-WAND framework by using it to study one dimension of co-creative communication-global versus local creative intent specification by the user-in the context of storytelling.


Artificial Intelligence for graphic design: how good are they in 2022?

#artificialintelligence

Have you worried that AI would automate away graphic design jobs? Instead of worrying, let's stay present and examine the latest results of graphic designs created by AI. The results mentioned below are from research papers published in top-tier computer science conferences or journals. In 2018, an artwork created by AI was sold in an auction at $432,500. By contrast, an unsung hero: an AI design system called Luban developed by Alibaba (an e-commerce giant), already delivered billions of banners/posters for customers! Recently, researchers proposed a new AI design system called Vinci[1].


the-rise-of-artificial-intelligence-driven-graphic-design

#artificialintelligence

Nowadays, it's more straightforward than at any other time to make a piece of graphic design, and this is particularly valid for logo design. These little graphic works will quite often be straightforward, with a moderate couple of components and restricted shadings. They effectively identify a brand and have not many prerequisites. Be that as it may, the design should be critical, extraordinary, and do the occupation for which they were designed. Hypothetically, logo design is entirely simple, to the point that anybody can deal with a DIY logo.